A memory cache, or more particularly, a CPU cache is a small, fast memory used by a central processing unit of a computer to reduce the average time to access memory. In contrast, regular or main memory is a relatively large memory, but is slow in comparison to the cache memory. The cache “caches” or stores copies of data from frequently used memory locations, thereby providing faster access to the frequently used memory locations.
In CPU cache systems, most memory accesses are from cached memory locations. As a result, an average latency of memory accesses can be closer to a latency of the cache memory than the latency of the main memory. Latency is essentially a delay between a CPU requesting a memory access (read or write) and the access completing. Latency is typically measured in bus cycles.
Cache memory works well for many applications. However, the cache memory and controls needed to access it can add considerable expense to a device. As a result, a designer may forego using cache memory in a product or device in order to reduce cost. This can cause the device or product to operate much slower than a similar device or product that incorporates a cache memory.